In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user
Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.
import os
import numpy as np # Importing numpy for Matrix Operations
import pandas as pd # Importing pandas to read CSV files
import matplotlib.pyplot as plt # Importting matplotlib for Plotting and visualizing images
import math # Importing math module to perform mathematical operations
import cv2 # Importing openCV for image processing
import seaborn as sns # Importing seaborn to plot graphs
from sklearn import metrics
# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD # Importing the optimizers which can be used in our model
from sklearn import preprocessing # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix # Importing confusion_matrix to plot the confusion matrix
from sklearn.preprocessing import LabelBinarizer
from keras.applications.vgg16 import VGG16
# Display images using OpenCV
from google.colab.patches import cv2_imshow # Importing cv2_imshow from google.patches to display images
from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
import random
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
from google.colab import drive
drive.mount('/content/drive/')
#loading data into a pandas dataframe
path="/content/drive/MyDrive/Great_Learning/files/Computer Vision/project_5/Labels.csv"
Labels_df=pd.read_csv(path)
# Load the image dataset (assuming it is a NumPy array)
image_path = '/content/drive/MyDrive/Great_Learning/files/Computer Vision/project_5/images.npy' # adjust the path as needed
images = np.load(image_path)
print(images.shape)
print(Labels_df.shape)
# Displaying the shapes of the loaded datasets to verify the data
print("Shape of Labels DataFrame:", Labels_df.shape)
print("Shape of Images Array:", images.shape)
There are 4750 RGB images of shape 128 x 128 X 3, each image having 3 channels.
cv2_imshow(images[5])
plt.imshow(images[5])
OpenCV cv2.imshow:
Displays the image in its default BGR format (colors may look incorrect as OpenCV does not convert BGR to RGB).
Image appears smaller and uses a pop-up window.
Matplotlib plt.imshow:
Displays the image correctly in RGB format.
The image appears larger and integrates well within Jupyter Notebook.
It shows axis scales, which are customizable.
# Converting the images from BGR to RGB using cvtColor function of OpenCV
for i in range(len(images)):
images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)
Why Conversion is Needed:
OpenCV loads images in BGR format by default.
Most deep learning frameworks (like TensorFlow, PyTorch) and visualization libraries (like Matplotlib) use the RGB format.
Code Explanation:
cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB) converts each image from BGR to RGB.
A loop is applied to process all images in the dataset.
Impact:
Ensures proper color representation for visualization and training.
Prevents color channel mismatch issues during training.
Efficiency:
def plot_images(images, labels_df):
"""
Plot random images with their corresponding labels.
Parameters:
- images: NumPy array of images (assumed shape: (num_images, height, width, channels)).
- labels_df: Pandas DataFrame containing the labels with 'Label' as the column name.
"""
rows = 3 # Number of rows
cols = 4 # Number of columns
fig = plt.figure(figsize=(10, 8)) # Figure size
for i in range(rows * cols):
random_index = np.random.randint(0, len(labels_df)) # Random index
ax = fig.add_subplot(rows, cols, i + 1) # Add subplot
ax.imshow(images[random_index]) # Plot the image
# Set the title as the corresponding label
ax.set_title(labels_df['Label'].iloc[random_index])
plt.tight_layout()
plt.show()
plot_images(images,Labels_df)
def plot_category_representatives(images, labels_df):
"""
Plot one representative image for each category.
Parameters:
- images: NumPy array of images (assumed shape: (num_images, height, width, channels)).
- labels_df: Pandas DataFrame containing the labels with 'Label' as the column name.
"""
categories = labels_df['Label'].unique() # Get unique categories
fig = plt.figure(figsize=(15, 10)) # Define figure size
for i, category in enumerate(categories):
# Find the first image index corresponding to this category
category_indices = labels_df[labels_df['Label'] == category].index
representative_index = category_indices[0]
# Add subplot for this category
ax = fig.add_subplot(3, 4, i + 1)
ax.imshow(images[representative_index]) # Plot the representative image
ax.set_title(category, fontsize=10) # Set the title as the category name
ax.axis('off') # Hide the axis for cleaner visualization
plt.tight_layout()
plt.show()
# Example usage
plot_category_representatives(images, Labels_df)
Visual Diversity:
Each plant category shows clear visual differences in shape, size, and orientation of leaves. For example:
"Fat Hen": Features small, upright leaves.
"Loose Silky-bent": Thin and grass-like structure.
"Shepherds Purse": Rounded, broader leaves.
Color Differences:
Most plants have green leaves, but the intensity of green varies:
Categories like "Common Chickweed" and "Cleavers" show a darker green.
"Common wheat" and "Loose Silky-bent" have slightly paler green tones.
Leaf Shape and Texture:
Categories differ in the leaf texture:
"Charlock": Broad leaves with noticeable structure.
"Black-grass": Very thin, elongated leaves resembling blades of grass.
"Sugar beet": Large leaves with prominent veins.
Surrounding Features:
Category Representation:
Practical Insights:
Leaf size, orientation, and texture are the key differentiators between plant types.
Categories like "Loose Silky-bent" and "Black-grass" may be harder to differentiate due to their similar thin leaf shapes.
sns.countplot(x=Labels_df['Label']) # Checking for data imbalance
plt.xticks(rotation='vertical') # Rotating the x-axis labels for better visibility
plt.show()
sns.countplot(Labels_df['Label'])
plt.xticks(rotation='vertical')
Based on the countplot of the Label column:
Class Imbalance Exists:
Some classes, such as "Loose Silky-bent", have significantly higher counts (over 600) compared to others.
Classes like "Shepherds Purse", "Cleavers", and "Black-grass" have much fewer samples, indicating underrepresentation.
Major Class:
Minor Classes:
Balanced Classes:
Risk of Model Bias:
# Count plot to check for data imbalance
sns.countplot(x=Labels_df['Label'], palette='viridis') # Use 'Label' column
plt.title('Class Distribution')
plt.xlabel('Plant Species')
plt.ylabel('Count')
plt.xticks(rotation='vertical') # Rotate labels for better visibility
plt.show()
Imbalance Exists:
Majority Class:
Minority Classes:
Distribution Varies:
As the size of the images is large, it may be computationally expensive to train on these larger images; therefore, it is preferable to reduce the image size from 128 to 64.
images_decreased = []
height = 64 # Define the height as 64
width = 64 # Define the width as 64
dimensions = (width, height)
for i in range(len(images)):
images_decreased.append(cv2.resize(images[i], dimensions, interpolation=cv2.INTER_LINEAR)) # Resize each image
plt.imshow(images[3])
plt.imshow(images_decreased[3])
Before Resizing:
After Resizing:
The image dimensions are reduced to 64x64 pixels.
The resized image appears blurred and pixelated due to loss of resolution.
# Applying Gaussian Blur to denoise the images
images_gb = [] # List to store images after Gaussian blur
for i in range(len(images)):
# Apply Gaussian blur with kernel size (3,3) and sigmaX = 0 (auto-calculated)
images_gb.append(cv2.GaussianBlur(images[i], ksize=(3, 3), sigmaX=0))
Split the dataset
Labels = Labels_df['Label'].values
X_temp, X_test, y_temp, y_test = train_test_split(np.array(images_decreased), Labels, test_size=0.1, random_state=42, stratify=Labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp, y_temp, test_size=0.1, random_state=42, stratify=y_temp)
print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)
Dataset Size and Splits:
The data has been split into training (80.1%), validation (9%), and test (10%) subsets. This ratio is appropriate for image classification tasks, where:
The training set is large enough for model learning.
Stratified Sampling:
Data Shapes:
Each image has dimensions (64, 64, 3), indicating they are resized RGB images.
Training set: 3847 images.
Validation set: 428 images.
Test set: 475 images.
Validation Set Proportion:
Test Set Size:
# Initialize LabelBinarizer
enc = LabelBinarizer()
# Fit and transform y_train, transform y_val and y_test
y_train_encoded = enc.fit_transform(y_train)
y_val_encoded = enc.transform(y_val)
y_test_encoded = enc.transform(y_test)
# Check the shape of the encoded labels
print(y_train_encoded.shape, y_val_encoded.shape, y_test_encoded.shape)
Label Encoding:
Consistency:
Shapes:
Readiness:
X_train_normalized = X_train.astype('float32') / 255.0
X_val_normalized = X_val.astype('float32') / 255.0
X_test_normalized = X_test.astype('float32') / 255.0
Normalization Process:
Pixel values are divided by 255.0 to scale them from the range [0, 255] to [0, 1].
This ensures the data is in a range suitable for neural networks, improving convergence.
Data Type:
Consistency:
Readiness:
# Clearing backend
backend.clear_session()
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Initializing a sequential model
model_1 = Sequential() # Initialize the model
# Adding the first conv layer with 64 filters and kernel size 3x3, padding 'same'
model_1.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))
# Adding MaxPooling to reduce the size of the output of the first conv layer
model_1.add(MaxPooling2D((2, 2), padding='same'))
# Creating two similar convolution and max-pooling layers with activation = relu
#model1.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
#model1.add(MaxPooling2D((2, 2), padding='same'))
model_1.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model_1.add(MaxPooling2D((2, 2), padding='same'))
# Flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model_1.add(Flatten()) # Flatten the 2D matrix into 1D
# Adding a fully connected dense layer with 16 neurons
model_1.add(Dense(16, activation='relu'))
model_1.add(Dropout(0.3)) # Dropout layer to avoid overfitting
# Adding the output layer with 12 neurons and softmax activation function for multi-class classification
model_1.add(Dense(12, activation='softmax'))
# Using the Adam optimizer
opt = Adam()
# Compiling the model with categorical cross-entropy loss and accuracy as the metric
model_1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model to see the details
model_1.summary()
history = model_1.fit(
X_train_normalized, y_train_encoded, # Training data
epochs=30, # Number of epochs (iterations over the dataset)
validation_data=(X_val_normalized, y_val_encoded), # Validation data for monitoring
batch_size=32, # Number of samples per gradient update
verbose=2 # Verbosity level, 2 for a progress bar per epoch
)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
accuracy = model_1.evaluate(X_test_normalized, y_test_encoded, verbose=2)
val_loss, val_accuracy = model_1.evaluate(X_val_normalized, y_val_encoded, verbose=2)
# Print Validation Accuracy
print(f"Validation Accuracy: {val_accuracy:.4f}")
train_loss, train_accuracy = model_1.evaluate(X_train_normalized, y_train_encoded, verbose=2)
print(f"Training Accuracy: {train_accuracy:.4f}")
# Here we would get the output as probablities for each category
y_pred= model_1.predict(X_test_normalized)
y_pred
model_name = "Base CNN Model"
# Predicting the output probabilities for each category
y_pred = model_1.predict(X_test) # Predict probabilities on test data
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg = np.argmax(y_pred, axis=1) # Convert probabilities to class labels
y_test_arg = np.argmax(y_test_encoded, axis=1)
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
Diagonal Values:
Misclassifications:
Class Imbalance:
Model Strength:
# Plotting the Confusion Matrix
confusion_matrix = tf.math.confusion_matrix(y_test_arg, y_pred_arg) # Compute the confusion matrix
# Plot the confusion matrix using Seaborn heatmap
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels')
ax.set_ylabel('True labels')
ax.set_title('Confusion Matrix')
ax.xaxis.set_ticklabels(list(enc.classes_), rotation=40) # Replace 'enc.classes_' with the actual label encoder classes if needed
ax.yaxis.set_ticklabels(list(enc.classes_), rotation=20) # Replace 'enc.classes_' with the actual label encoder classes if needed
plt.show()
# Assuming y_test_arg and y_pred_arg contain the true and predicted class indices, respectively
cr = metrics.classification_report(y_test_arg, y_pred_arg)
# Print the classification report
print(cr)
Class Imbalance Issues:
Best Performance on Class 3:
Moderate Performance on Few Classes:
Reducing the Learning Rate:
Hint: Use ReduceLRonPlateau() function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.
# Code to monitor val_accuracy
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
patience=3,
verbose=1,
factor=0.5,
min_lr=0.00001)
Remember, data augmentation should not be used in the validation/test data set.
# Clearing backend
backend.clear_session()
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Data Augmentation
train_datagen = ImageDataGenerator(
rotation_range=20, # Rotation angle range to randomly rotate images during training
fill_mode='nearest' # Fill the pixels in the new area created by rotation
)
# Initializing a sequential model
model_2 = Sequential()
# Adding the first convolution layer with 64 filters, kernel size 3x3, and padding 'same'
model_2.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))
# Adding max pooling to reduce the size of the output of the first convolution layer
model_2.add(MaxPooling2D((2, 2), padding='same'))
# Adding second convolution and max pooling layers
model_2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model_2.add(MaxPooling2D((2, 2), padding='same'))
# Adding BatchNormalization for faster convergence
model_2.add(BatchNormalization())
# Flattening the output of the convolution layers after max pooling to make it ready for dense layers
model_2.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
model_2.add(Dense(16, activation='relu'))
# Adding dropout with dropout_rate=0.3 to reduce overfitting
model_2.add(Dropout(0.3))
# Adding the output layer with 12 neurons and softmax activation function for multi-class classification
model_2.add(Dense(12, activation='softmax'))
# Initializing the Adam optimizer
opt = Adam()
# Compiling the model with categorical crossentropy loss function and accuracy as the metric
model_2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model_2.summary()
# Defining epochs and batch_size for training
epochs = 30
batch_size = 64
# Training the model on the augmented data
history_0 = model_2.fit(
train_datagen.flow(X_train_normalized, y_train_encoded, batch_size=batch_size, shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized, y_val_encoded),
verbose=1,
callbacks=[learning_rate_reduction]
)
plt.plot(history_0 .history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
# Plotting the training and validation loss
plt.plot(history_0 .history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
accuracy = model_2.evaluate(X_test_normalized, y_test_encoded, verbose=2)
val_loss, val_accuracy = model_2.evaluate(X_val_normalized, y_val_encoded, verbose=2)
# Print Validation Accuracy
print(f"Validation Accuracy: {val_accuracy:.4f}")
train_loss, train_accuracy = model_2.evaluate(X_train_normalized, y_train_encoded, verbose=2)
print(f"Training Accuracy: {train_accuracy:.4f}")
model_name = "CNN Model with Data Augmentation"
# Here we would get the output as probablities for each category
y_pred=model_2.predict(X_test_normalized)
y_pred
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
Dominant Predictions for Certain Classes:
The model performs relatively well for Class 6 (57 correct predictions) and Class 3 (54 correct predictions).
These classes seem to have better recall compared to others.
Misclassifications:
Class 0 is heavily misclassified, with many instances incorrectly predicted as Class 6 (22 instances).
Similar issues exist for other classes, such as Class 7 and Class 5, with predictions scattered across other classes.
Class Imbalance Impact:
Diagonal Dominance:
# Complete the code to obtain the output probabilities
y_pred = model_2.predict(X_test_normalized) # Predicting the output probabilities for test data
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg = np.argmax(y_pred, axis=1) # Converting probabilities to categorical predictions
y_test_arg = np.argmax(y_test_encoded, axis=1) # Converting true labels to categorical values
# Plotting the Confusion Matrix using confusion_matrix() function which is predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg, y_pred_arg) # Generating the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True, # Annotating the confusion matrix with the numbers
linewidths=.4,
fmt="d", # Displaying the values as integers
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels')
ax.set_ylabel('True labels')
ax.set_title('Confusion Matrix')
ax.xaxis.set_ticklabels(list(enc.classes_), rotation=40) # Assuming `enc.classes_` has the class names
ax.yaxis.set_ticklabels(list(enc.classes_), rotation=20)
plt.show()
# Complete the code to generate the classification report
cr = metrics.classification_report(y_test_arg, y_pred_arg) # Generate the classification report
# Print the classification report
print(cr)
Overall Accuracy:
Class Performance:
Strong Performance:
Weak Performance:
Class 0 has the lowest performance, with a precision of 0.33 and a recall of 0.12, suggesting that it is difficult for the model to identify and correctly predict this class.
Class 4 also struggles, with very low recall (0.05) and an F1-score of 0.08, indicating poor detection of this class.
Recall vs. Precision:
High Recall: Class 6 has a high recall of 0.88 but a lower precision of 0.55, suggesting that while the model is good at detecting this class, it also produces many false positives.
Balanced Metrics: Classes 3, 1, and 11 show a good balance between precision and recall, leading to high F1-scores.
Macro vs. Weighted Average:
# Load the pre-trained VGG16 model without the top layer
vgg_model = VGG16(weights='imagenet', include_top=False, input_shape=(64, 64, 3))
# Freeze all the layers in the VGG16 model
for layer in vgg_model.layers:
layer.trainable = False
# Build your custom model
new_model = Sequential()
# Add the VGG16 convolutional base
new_model.add(vgg_model)
# Add custom layers on top of the VGG16 base
new_model.add(Flatten())
new_model.add(Dense(32, activation='relu'))
new_model.add(Dropout(0.2))
new_model.add(Dense(16, activation='relu'))
# Calculate num_classes
num_classes = y_train_encoded.shape[1] # Ensure y_train_encoded is properly encoded
# Add the output layer
#new_model.add(Dense(10, activation='softmax'))
new_model.add(Dense(num_classes, activation='softmax'))
# Compile the model
opt = Adam()
new_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Verify the model structure
new_model.summary()
# Train the model
epochs = 30
batch_size = 64
history_vgg16 = new_model.fit(
train_datagen.flow(X_train_normalized, y_train_encoded, batch_size=batch_size, seed=42, shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized, y_val_encoded),
verbose=1
)
# Plot accuracy
plt.plot(history_vgg16.history['accuracy'], label='Train Accuracy')
plt.plot(history_vgg16.history['val_accuracy'], label='Validation Accuracy')
plt.legend()
plt.show()
# Plot loss
plt.plot(history_vgg16.history['loss'], label='Train Loss')
plt.plot(history_vgg16.history['val_loss'], label='Validation Loss')
plt.title('Training and Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()
accuracy = new_model.evaluate(X_test_normalized, y_test_encoded, verbose=2)
loss, accuracy = new_model.evaluate(X_test_normalized, y_test_encoded, verbose=2)
print(f"Test Loss: {loss}")
print(f"Test Accuracy: {accuracy}")
val_loss, val_accuracy = new_model.evaluate(X_val_normalized, y_val_encoded, verbose=2)
# Print Validation Accuracy
print(f"Validation Accuracy: {val_accuracy:.4f}")
train_loss, train_accuracy = new_model.evaluate(X_train_normalized, y_train_encoded, verbose=2)
print(f"Training Accuracy: {train_accuracy:.4f}")
# Model name
model_name = "Transfer Learning Model"
y_pred = new_model.predict(X_test_normalized)
y_pred
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
Improvement in Class 6:
Misclassification Patterns:
Class 0 is misclassified often as Class 3 (16 times), highlighting a need to distinguish these two classes better.
Class 5 has significant misclassifications across multiple classes, with only 16 correctly predicted.
Class 3 Performance:
Diagonal Dominance:
# Complete the code to obtain the output probabilities
y_pred = new_model.predict(X_test_normalized) # Predicting the output probabilities for test data
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg = np.argmax(y_pred, axis=1) # Converting probabilities to categorical predictions
y_test_arg = np.argmax(y_test_encoded, axis=1) # Converting true labels to categorical values
confusion_matrix = tf.math.confusion_matrix(y_test_arg, y_pred_arg) # Generating the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True, # Annotating the confusion matrix with the numbers
linewidths=.4,
fmt="d", # Displaying the values as integers
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels')
ax.set_ylabel('True labels')
ax.set_title('Confusion Matrix')
ax.xaxis.set_ticklabels(list(enc.classes_), rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_), rotation=20)
plt.show()
# Assuming y_test_arg and y_pred_arg contain the true and predicted class indices, respectively
cr = metrics.classification_report(y_test_arg, y_pred_arg)
# Print the classification report
print(cr)
Overall Accuracy:
Class-Level Observations:
Strong Performance:
Class 6 has the highest recall (0.92) and a good F1-score (0.71), showing the model's strong ability to detect instances of this class.
Class 7 has a high precision (0.85), but its recall is only 0.50, indicating good specificity but struggles in identifying all true positives.
Weak Performance:
Class 4 has zero scores for precision, recall, and F1, indicating that the model completely fails to predict this class.
Classes 9 and 0 also perform poorly, with low recall and F1-scores, highlighting challenges in identifying these classes.
Imbalance Between Precision and Recall:
Some classes, such as 10, have significantly better recall (0.74) than precision (0.40), suggesting many false positives.
Conversely, classes like 7 have higher precision but lower recall, indicating the opposite issue.
Macro vs. Weighted Averages:
accuracy = model_2.evaluate(X_test_normalized, y_test_encoded, verbose=2)
val_loss, val_accuracy = model_2.evaluate(X_val_normalized, y_val_encoded, verbose=2)
# Print Validation Accuracy
print(f"Validation Accuracy: {val_accuracy:.4f}")
train_loss, train_accuracy = model_2.evaluate(X_train_normalized, y_train_encoded, verbose=2)
print(f"Training Accuracy: {train_accuracy:.4f}")
# Here we would get the output as probablities for each category
y_pred=model_2.predict(X_test_normalized)
y_pred
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[2])
plt.show()
# Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model_2.predict((X_test_normalized[2].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[2]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[33])
plt.show()
# Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model_2.predict((X_test_normalized[2].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[33]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[59])
plt.show()
# Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model_2.predict((X_test_normalized[2].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[59]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[36])
plt.show()
# Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model_2.predict((X_test_normalized[2].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[36]) # using inverse_transform() to get the output label from the output vector
Performance Metrics:
Training Accuracy: 78.42%
Validation Accuracy: 73.13%
Test Accuracy: 73.68%
The model generalizes well, as the gap between training, validation, and test accuracies is minimal, indicating no overfitting.
Prediction Probabilities:
The model outputs probabilities for each category, suggesting it is suitable for multi-class classification.
Higher probabilities for correct classes demonstrate the model's confidence.
Confusion Matrix:
Diagonal Dominance: The matrix's diagonal contains the highest values, indicating accurate predictions for most classes.
Misclassifications: Off-diagonal entries represent misclassified samples.
Certain categories may need further augmentation or fine-tuning.
Balanced Performance: No significant bias toward any specific class, suggesting improved handling of class imbalance.
Improvements Over Other Models:
Prediction Consistency Issue:
All the images are predicted as 'Small-flowered Cranesbill' irrespective of their true labels:
True Label: Small-flowered Cranesbill → Correct Prediction
True Label: Cleavers, Common Chickweed, Shepherds Purse → Incorrect Prediction.
Model Bias:
Evaluation of Predictions:
pd.DataFrame({'Models':['Base CNN Model','CNN Model with Data Augmentation','Transfer Learning Model'],'Train Accuracy':['61%','78%','61%'],'Validation Accuracy':['53%','73%','50%'],'Test Accuracy':['51%','74%','53%']})
We can observe from the confusion matrix of all the models that our CNN Model with Data Augmentation model was the best model because it predicted the majority of the classes better than the other models.
The test accuracy of the CNN Model with Data Augmentation model is 73%. Data Augmentation has also helped in improving the model.
Performance:
Insight:
Performance:
Insight:
Performance:
Insight:
Reason: